As a precaution. This function might get used for some long persisted
file descriptor and we need good defaults.

libutil/unix: Add O_CLOEXEC to openDirectory

this is a painful change. we should really add EvalState or EvalMemory
as an argument to various functions as we need it, but because we want
to preserve the stablity API, we hack it in as a field of nix_value.

Introduce GitAccessorOptions

builtins.deepSeq on deeply nested structures (e.g., a linked list with
100,000 elements) caused an uncontrolled OS-level stack overflow with
no Nix stack trace.

Fix by adding call depth tracking to forceValueDeep, integrating with
Nix's existing max-call-depth mechanism. Now produces a controlled
"stack overflow; max-call-depth exceeded" error with a proper stack
trace.

Closes: NixOS#7816

When deepSeq encounters an error while evaluating a list element, the
error trace now includes the list index, making it easier to locate
the problematic element.

Similar to the deepSeq fix, toJSON on deeply nested structures caused
an uncontrolled OS-level stack overflow.

Fix by adding call depth tracking to printValueAsJSON.

Those can never be nullptr, so we should use the type system
to ensure this invariant.

This is necessary to make pausing/unpausing possible in a follow-up commit.

…ad thread

Instead of naively stalling the download thread we can instead stop the transfer.
This allows the other multiplexed connections to continue downloading (and unpacking),
if the result of the download gets piped into a GitFileSystemObjectSink.

Prior art in lix project:

- https://git.lix.systems/lix-project/lix/commit/4ae6fb5a8f0d456b8d2ba2aaca3712b4e49057fc
- https://git.lix.systems/lix-project/lix/commit/12156d3beb8a16c0e2e8cf7180e1fbf27280a669

This patch is very different from the lix one, since we are using a decompression sink
in the middle of the pipeline but the co-authored-by is there since I was motivated to
implement this by looking at the lix side of things.

Co-authored-by: eldritch horrors <[email protected]>

Since the root cause (the lack of backpressure control) has
been fixed in the previous commit we can revert the change from
8ffea0a and make the default size much
smaller.

libstore/filetransfer: Pause transfers instead of stalling the download thread

Updated documentation to clarify that building without optimization can lead to faster builds.

Clarify build options in debugging documentation

On FreeBSD, sysctl(KERN_PROC_PATHNAME) returns a null-terminated
string with pathLen including the terminator. This causes Nix to
fail during manual generation with:

  error:
         … while calling the 'concatStringsSep' builtin
           at /nix/var/nix/builds/nix-63232-402489527/source/doc/manual/generate-settings.nix:99:1:
             98| in
             99| concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
               | ^
            100|

         error: input string '/nix/store/gq89cj02b5zs67cbd85vzg5cgsgnd8mj-nix-2.31.2/bin/nix␀'
                cannot be represented as Nix string because it contains null bytes

The issue occurs because generate-settings.nix reads the nix binary
path from JSON and evaluates it as a Nix string, which cannot contain
null bytes. Normal C++ string operations don't trigger this since they
handle null-terminated strings correctly.

Strip the null terminator on FreeBSD to match other platforms (Linux
uses /proc/self/exe, macOS uses _NSGetExecutablePath).

Credit: @wahjava (FreeBSD ports and Nixpkgs contributor)

…rminator

fix(FreeBSD): remove null terminator from executable path

`deepSeq`, json: handle stack overflow, report list index

Without this dependency, e.g. `meson compile nix-instantiate`
would produce a broken symlink, or the `nix` it points to may be
stale.
With the dependency in place, `meson compile nix-instantiate`
produces a reliable outcome.

This partially reverts commit bc6b9ce.

This transformation is unsound and thread unsafe. Internal libgit2
structures must *never* be shared between threads. This causes
internal odb corruption with e.g.:

nix flake prefetch-inputs:

error:
       … while fetching the input 'github:nixos/nixpkgs/89c2b2330e733d6cdb5eae7b899326930c2c0648?narHash=sha256-Stk9ZYRkGrnnpyJ4eqt9eQtdFWRRIvMxpNRf4sIegnw%3D'

       error: adding a file to a tree builder: failed to insert entry: invalid object specified - upload-image.sh
error:
       … while fetching the input 'github:NixOS/nixpkgs/a8d610af3f1a5fb71e23e08434d8d61a466fc942?narHash=sha256-v5afmLjn/uyD9EQuPBn7nZuaZVV9r%2BJerayK/4wvdWA%3D'

       error: adding a file to a tree builder: failed to insert entry: invalid object specified - outline.nix
double free or corruption (!prev)

Thread 21 "nix" received signal SIGABRT, Aborted.

src/nix: Make meson compile <cmdlet> valid

libexpr: plug ExprCall memory leak

libfetchers: Don't have a single shared tarball cache

libexpr: use allocBytes() to allocate StringData

This matches what we just did for `nix path-info`, and I hope will allow
us to avoiding any more breaking changes to this command for the
foreseeable future.

(cherry picked from commit 0f18076)

Tagging release 2.33.0

This was ignored for some reason.

Unfortunately previous tarball caches had loose objects written to
them and subsequent switch to thin packfiles. This results in possibly
broken thin packfiles when the loose objects backend is disabled. Thin
packfiles do not necessarily contain the whole closure of objects.
When packfilesOnly is true we end up with an inconsistent state where
a tree lives in a packfiles which refers to a blob in the loose objects
backend.

In the future we might want to nuke old cache directories and repack
the tarball cache.

If the total number of file buffers exceeds maxBufSize, we switch to
writing synchronously.